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ABSTRACT 

This fourth volume of ? four-volume evaluation of the 
1988-89 New York City School Community Education Program (also known 
as the Umbrella Program) comprises reports evaluating six innovative 
projects that offered a variety of educational experiences to 
participants. Three of these projects were designed for 
prekindergart'en children; the other three were designed to teach 
elementary school students health maintenance concepts, improve their 
acceptance rates to special high schools, and foster career 
awareness. Evaluation sources included student preprogram and 
postprogram test outcomes, writing samples, teacher and student 
questionnaires, and the number of acceptances of participants into 
special hi^h schools. Overall, the program was not as successful in 
meeting it^ stated objectives as in previous years. Each report 
contains a brief project overview, describes the research 
ip^thodology, presents the findings, and provides recommendations for 
improvement. The following projects are evaluated: (1) High School 
Preparation and Choice; (2) the East Harlem Pre-Kindergarten Center; 
(3) the Brook.lyn College Prekindergarten Center; (4) the Publishing 
Activity Center; (5) the Basic Skills After School Program; and (6) 
the Traveling Body Shop. Statistical data are included on uine 
tables. Each report also includes examples of evaluation instruments. 
(FMW) 
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SCHOOL COMMUNITY EDUCATION PROGRAM 
IN NEW YORK CITY ^ 
EVALUATION SUMMARY ^ 



BACKGROUND 

The School Community Education Program (also known as the 
Umbrella Program) , administered by the Division of Curriculim and 
Instruction, provides a variety of educational and training 
experiences to a wide range of participants, including pre- 
schoolers and their parents, and elementary, intermediate, and 
high school students, teachers, and supervisors. The program 
consists of 37 different projects designed to provide innovative 
solutions to local educational and school problems. Ten p^^^jects 
provide basic skills, English as a Second Language, and computer 
literacy instruction; ten focus on social issues and 
environmental studies; seven offer staff development workshops; 
five involve curriculum development, and three are designed for 
prekindergarten children. The remaining projects provide 
participants with a variety of educational experiences. 



POPULATION SERVED 

In 1988-89, the program served some 25,000 students, 
primarily elementary school pupils. In addition, the program 
served 1,100 teachers and supervisors and 100 prekindergarten 
children, as well as neighborhood adults in the 32 community 
school districts and selected high schools. Each project 
established different selection criteria for program 
participation. 



PROGRAM OBJECTIVES _ 

Although program objectives were designed for each specific 
project and therefore varied, most focused on increasing the 
competence of project participants through mastery of specific 
skills and abilities. Host objectives also set quantitative 
criteria to be met by a minimum percentage of participants for 
the program to be considered successful. 



*This summary is based on the final evaluation report of the 
School Community Education Program in New York City 1988-89, 
prepared by the Office of Research, Evaluation, and Assessment/ 
Instructional Support Evaluation Unit« 



f!VAT.TTATTnW MET HODOLOGY 



The evaluation of the program is based on a number of data 
sources: student performance outcomes on standardized or 
project-developed tests, pupil writing samples, teacher and 
student survey questionnaires, number of acceptances to special 
high schools, and review of five curriculum documents. These 
manuals and lesson. plans were sent to different tmits of the New 
York City Board of Education's Division of Curriculum and 
Instruction for evaluation. Preprogram and pestprogram test 
outcomes were compared to deteimine mean differences and, when 
appropriate, correlated t- tests and effect sizes were also 
computed to establish statistical significance and educational 
meaningfulness, respectively. The percentage of participants 
meeting quantitative project-set criteria for success was also 
determined . 



FINDINGS 

The 1988-89 evaluation findings indicate that the School 
community Education Program was not as successful as it had been 
in previous years. Only 15 projects met their stated objectives, 
compared to 19 in 1987-88. In general, those projects providing 
staff development training and curriculum development were the 
most successful. In addition, two projects that provide remedial, 
instruction ^Harlem School-Community Tutorial Project, and 
Mathematics Improvement Program) were also found to be 
particularly successful. The evaluation also showed that 
although some projects met their objectives, these results should 
be treated with caution because of the vagueness of the 
objectives or because the evaluation instruments could not 
adequately measure project impact. This is a particular problem 
shared by staff development projects that seek to measure teacher 
ability to implement specific teaching skills in the classroom 
without including instruments which measure these skills. 

Four projects were successful in meeting one of their 
objectives, yet unsuccessful in meeting a second objective. 
Sixteen projects did not meet their evaluation objectives, and 
two projects could not be evaluated because test data were 
lacking. As indicated in previous years' evaluations, a few of 
these projects need extensive modifications, such as revision of 
testing instruments to avoid ceiling effect, development of 
project activities appropriate for different grade levels, or 
establishment of more stringent participant selection criteria. 
Most of the unsuccessful projects, however, failed to meet their 
objectives because their criteria for success were too stringent 
or because the testing instru^jent could not adequately measure 
pioject objectives. In some of these projects, participants 
achieved large mean gains, but the percentage of successful 
participants remained below the percentage established in the 
project-set criterion for success. In some cases, this criterion 
was beyond what could be reasonably expected of program 



f participants. 

r- 

hi,, RECOMMENDATIONS 

p- * In addition, to the recommendations made for each project, 

the following suggestions are made for the overall improvement of 
the school Community Education Program: ^ 

• Closely *aonitor those projects that fail to meet their 
stated objectives, 

' * Assist project staff in making necessary project 

modifications such as the revision of project activities, 

I revision or replacement of testing instruments, 

establishment of adequate selection criteria of 
participants, or amendments in project objectives. 
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INTRODUCTION 



In 1988-89, the New York City Public Schools received 
$2,375,000 in funding from the New York State Legislature to 
operate the School Community Education program (also known as the 
Umbrella program) . It consisted of 37 different projects 
designed to provide innovative solutions to local educational and 
school programs. 

The program provided services to about 25,000 participants 
in 32 community school districts and selected high schools. 
While most of these participants were elementary school students, 
the program also served some 1,000 intermediate and high school 
students, 100 preschtfol children, and 1,100 teachers and 
supervisors. Some projects also included parenting components 
and/or sought to involve the parents of participating students in 
project activities. 

Evaluation reports are presented in four volumes. Volume I 
contains evaluations of ten projects that provided reading, 
mathematics, writing, English as a Second Language, and computer 
literacy instruction. Volume II includes evaluations of nine 
projects on social, echnic, and environmental studies. Four of 
these projects also provided staff development workshops. Volume 
III contains evaluations of seven staff development and five 
curriculxm development projects. The remaining six projects, 
presented in Volume IV, offered a variety of educational 
experiences to participantF , Three of these projects were 
designed for prekindergarten children, and the other three 
projects were .Ijsigned to teach students health maintenance 
concepts, to improve their acceptance rate to special high 
schools, and foster career awareness among students. 

Each report contains a brief project overview, describes the 
avaluation methodology, presents the findings, and provides 
recommendations for improvement. The reports are listed in order 
of budgeted function number in the Table of Contents. 
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#5001-'48-93412 

HIGH SCHOOL PREPARATION AND CHOICE, 1988-89 

School -Coxnmunity Education Program 
Program Administrator: M*. Morris Speiser 
Project Coordinator: Robert Rogers 

Prepared by^^^ 
Office of Research, Evaluation^ and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The High School Preparation and Choice project is designed 
to prepare seventh and eighth grade students in Community School 
District (C.S.D.) 17 for acceptance at special high schools. The 
goal of the project is to provide students who show academic 
potential and interest in continuing their education with skills 
necessary to succeed in the competitive high school selection 
process. The New York State Legislature provided $21 thousand to 
fund the project. 

In 1988-89, 140 students from seven intermediate schools 
participated in the program. School principals, guidance 
counselors, and teachers selected 20 pupils in each school from 
those who planned to apply to a special high school. The 
students had a range of achievement levels in reading and 
mathematics. 

Participants met with project teach<ars for a maximum of 
three after-school sessions of two hours each every week. Group 
instruction was supplemented with small group and individual 
tutoring sessions. Instructional activities were designed to 
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familiarize students with high school programs, entrance 
requirements and application procedures. Participants took 
practice high school admission tests and received individualized 
advice regarding career and high school choices. The project 
objective was for participants to show a higher acceptance rate 
to special high schools than a comparable group of non- 
participating students applying to similar high schools. 

EVALUATION METHODOT.iOGY AND FINDINGS 

The primary data source consisted of students participating 
in the project and those who were accepted by special high 
schools, and the same number of non-participating students that 
were also accepted by these schools (see Table 1) • Students in 
the control group were selected by the project coordinator and 
displayed a range of reading scored on the Progress of Reading 
Power (P.R.P.) comparable to the students participating in the 
project. The special high schools were Stuyvesant, Brooklyn 
Technical, Bronx High School of Science, Fiorello H. La Guardia, 
and High School of Performing Arts. 

Thirty percent of participants gained acceptance to special 
high schools. Only three percent of students who did not receive 
project services were accepted by these schools. Analyzed by 
school, data show that P.S. 189 had the largest percentage (85 
percenc) of successful students. 
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TABLE 1 

Number of Special High School Acceptances 
of Project Participants and Non-Project Participants, 

by School 

High School Preparation and Choice, 1988-89 







ProTGct Particioants 




Non- 


■Proiect Particinants 


«9 Wilww J. 


K 

A. 


Number of 


Percent 


School 


N 


Number of 
Acceotiances 


Percent 


I.S. 61 


20 


2 


10.0% 


I.S. 


61 


20 


2 


10.0% 


I,S. 210 


20 


0 


0.0 


I.S. 


210 


20 


0 


0.0 


I.S. 246 


20 


2 


10.0 


I.S. 


246 


20 


1 


5.0 


I.S. 320 


20 


10 


50.0 


I.S. 


320 


20 


1 


5.0 


I.S. 390 


20 


6 


30.0 


I.S. 


390 


20 


1 


5.0 


I.S. 391 


20 


5 


25.0 


I.S. 


391 


20 


0 


0.0 


P.S. 189 


20 


17 


85.0 


P.S. 


189 


20 


0 


5.0 


TOTAL 


140 


42 


30.0% 


TOTAL 


14 C 


5 


3.6% 



* A larger percentage of project participants (30 percent) than 
non-project participants. (3.6 percent) gained acceptance to 
special high schools. 
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The High School Preparation and Choice project was a 
successful program with a positive impact on participating 
students. A comparison of their performance with that of the 
control group of students who did not participate in the project 
shows that intensive instruction and tutoring helped project 
participants gain acceptance to special high schools at a higher 
rate that non-project participants. 

Although the project was successful overall, certain sites 
performed better than others. P.S. 189 had the largest 
acceptance rate (85 percent) whereas I.S. 210 had no project 
participants accepted to special high schools. It is unclear 
which factors contributed to the difference between sites. In 
the future, it may be helpful to examine the differences and 
similarities between the sites to understand why some sites 
perform better than others. 

It is also recommended that the number of contact hours for 
eighth grade students be increased by employing staff before the 
regular school day and also after school, wherever possible. 
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EAST HARLEM PRE-KINDERGARTEN CENTER, 1988-89. 

School-Community Education Program 
Program Administrator: M* Morris Speiser 
Project Coordinator: Evelyn Castro 

Prepared By: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The East Harlem Pre-Kindergarten Center provides learning 
experiences to three- and fcur-year-old children in the East 
Harlem area. The program is designed to build specific skills in 
areas that include language arts, problem solving, sensory 
awareness, fine- and gross-motor coordination- and physical 
development. The project also seeks to develop a positive self- 
image among participants. 

The preschool program operated at P.S. 171 in Community 
school District (C.S.D.) 4, served some 15 children. The 
children attended sessions five days a week and were selected 
from families whose older children were low achievers in school 
and showed poor socialization. Instructional activities focused 
on listening and speaking, first-hand experiences, learning the 
alphabet, and experimenting with discovery and tactile materials. 
The program objective for 1988-89 was for participants to 
demonstrate a 20 percent increase in readiness and developmental 
skills as measured by their performance on the Developmental 
Checklist. 
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Project staff consisted of one early-childhood teacher and a 
paraprofessional. In addition, one family associate served as a 
liaison between the school and the community to help parents 
support and reinforce program activities* Parents also attended 
monthly workshops involving information on health, welfare, 
housing, child care, and child development. The New York State 
Legislature contributed $44 thousand to cover program 
expenditures, and purchase instructional equipment and supplies* 
EVALUATION METHODOLOGY 

Evaluation activities focused on the analysis of the 
participants' pre- and posttest scores on the developmental 
checklist (see appendix A) . The Developmental Checklist is 
designed to assess four levels of child development: personal and 
social behavior, physical development, intellectual development, 
and creative expression. The checklist consists of 48 items. 
Teachers rate the participants on a scale of 1 to 3, with l=the 
child does not exhibit this behavior and 3=the child exhibits 
this behavior most of the time. 
FINDINGS 

Pretest and posttest scores were submitted for 15 children. 
The results of the analyses are presented in Table 1. Mean 
pretest raw score was 137 points (73.7 percent correct 
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TABLE 1 

Children's Mean Raw Scores* 
on the Developmental Checklist 
East Karlem Pre-Kindergarten Center, 1988-89 



Pretest Mean 
Raw Percent 
Score Correct 



Posttest Mean 
Raw Percent 
Score Correct 



Mean Gain 
Raw Percent 
Score Correct 



Students 

Meeting Criterion 
(N=6) 

Students Not 
Me^.ting Criterion 
(w-9) 



125.3 67.4% 



144.8 77.9 



168.3 90.5% 



174.9 94.0 



43.0 



23.1% 



30.1 16.2 



Total 
(N=15) 



137.0 73.7 



172.3 92.6 



35.3 19.0 



'Perfect raw score=186. 



Forty percent of the participants met or surpassed the 
project-set criterion of a 20 percent increase. 



93416 



Children who did not participate in the program. The age period 
of three to four is a time of growth, both physically and 
mentally, for all children, and although a prekindergarten 
program may help promote that growth, it is not possible to 
examine its impact beyond normal development without a comparison 
group . 

Finally, the present objective should be revised in more 
concrete teirms. At present, the objective is ambiguously worded 
stating that "participants will achieve a 20 percent gain.»« It 
is unclear whether this refers to each child individually or to 
the average score received by the group of children • It is not 
reasonable to expect all children to receive gains. Therefore it 
is recommended that the objective be revised to a specific 
quantitative goal, for example "80 percent of participants will 
achieve a 20 percent gain." 
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t.Trxrrr«m™ APPENDIX A 

EVALUATION 93416 
DEVELOPMENTAL • CHECKLIST 



Child Date of Birth 



Pre-kindergarten Class 



Pre-kindergarten Teacher 



Not Some of Most of 
yet the time the time 



Personal and Social Behavior 

1. Buttons coat or jacket without 
assistance 



2. 


ZIds c^at or iacket without 




assistance 


3. 


Removes and puts on rainboots 




without assistance (not tied) 


4. 


Fastens own shoes (not tied) 


5. 


Knows full name 


c 

o « 


IVnvWb nUlUc auQLcbb 


7. 


Replaces toys and materials 


8. 


Evaluates own behavior 


9. 


Makes friends with other children 


10. 


Shows reasonable self control 


11. 


Follows class rules 


12. 


Accepts changes in routines 


13. 


Shares witn other children 


14. 


Knows and uses names of adults 




in classroom 


15. 


Tries new things 


16. 


Is able to take turns 



0017F 
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Some of 
the time 



Physical Development 

■r- -* I - - I 

17. Uses large muscles to: 



0017F 



hop 



3ump 



run 

gallop 



throw a ball 



18« Uses small muscles to: 



cut with scissors 



paste 



draw with crayons 



manipulate toys 



play with puzzles 



work with clay 



19 • Walks up and down steps using 
two feet alternately 



20. Climbs, slideS/ swings on outdoor 
equipment 



21. Is able to place pegs in holes 



22. Strings one-inch beads 



23. Erects simple block structures 



93416 



Not Some of Most of 
yet the time the time 



38. 


Shows an interest in books and 
stories 




39. 


Recites simple poems and 
finger plays 




40. 


Memorizes and sings simple songs 




41. 


Participates in group discussions 
ard meetings 




Creative Expression 




42. 


Sings songs 




43. 


Experiments with paints 




44. 


Dramatizes simple stories and poems 




45. 


Explores the sounds of rhythm 
instruments 




46. 


Responds to music through movement 




47. 


Models with clay 




48. 


User> a variety of tools in drawing 
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#5001-48-93419 

BROOKLYN COLLEGE PREKINDERGARTEN CENTER, 1988-89 

School-Coininunity Education Program 
Program Administrator: M. Morris Speiser 
Project Coordinator: Paula Longo 

Prepared By: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The Brooklyn College Tutorial Center provides children with 
experiences and opportunities to enhance their intellectual, 
social, emotional, anJ physical development, and provides them 
with the knowledge of basic concepts necessary fcr achievement in 
school. The program also includes a parenting workshop for 
participants' parents to learn to deal more effectively with 
their cbildrei?. The New York State Legislature contributed $114 
thousand to cover personnel expenses and to purchase educational 
supplies. 

A total of 60 children participated in the program in 1988- 
89. All participants were residents of Brooklyn and were between 
the ages of two years and nine months to three years and eight 
months. Children were selected for participation by a lottery. 
Consideration was given to those children development ally able to 
benefit from the program and attempts were made to establish an 
ethnically diverse population. The parenting workshops were open 
to participants' parents. 

Instruction focused on the development of language arts, and 
social and physical skills, emphasizing divergent thinking, 
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creativity, fluency, and flexibility. In addition, children were 
provided with individual library cards and encouraged to take out 
a book each week from the "circulating library.'' Each class was 
staffed by one full-time teacher and a paraprofessional. 

The program objective was for participants to Increase by at 
least 20 percent their knowledge of basic concepts necessary for 
achievement in school, as measured by their performance on the 
Developmental Checklist. 

The parenting workshop series^ was designed to broaden 
parents* knowledge of child development, provide instruction in 
skills such as reflective listening, examine reasons for and 
means of controlling children's misbehavior, and to encourage 
parents to read to their children. Workshops also included 
lectures on topics such as nutrition. A program objective for 
this component of the program was not specified. 
EVALUATION METHODOLOGY 

Evaluation activities focused on the analysis of the 
participants pre- and posttest scores on the developmental 
checklist (see Appendix A) . The Developmental Checklist is 
designed to assess four levels of child development: personal and 
social behavior, physical development, intellectual development, 
and creative expression. Teachers rate the participants for each 
item on a scale of 1 to 3, with l=the child does not exhibit this 
behavior, to 3=the child exhibits this behavior most of the time. 

2 
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FINDINGS 

Complete test scores were siibmitted for 60 children. The 
results of the analyses are presented in Table 1. Overall, mean 
pretest raw score was 107.2 points (58.6 percent correct 
responses), and mean posttest raw score was 159.5 points (87.2 
percent correct), for a mean gain of 52.3 points or 28.6 percent 
increa&e. Overall, 95 percent of participants met or surpassed 
the project-set criterion of a 20 percent increase on posttest. 
CONCLUSIONS AND RECOMMENDATIONS 

In 1988-89, the Brooklyn Prekindergarten Center was 
successful in improving chilJren's performance on the 
Developmental Checklist. Ninety-five percent of the participants 
achieved gairs of at least 20 percent. This is -a higher 
percentage of successful students than in previous years. Only 
three participants did not meet this project-set criterion and 
these children received high pretest scores which leave little 
room for a 20 percent gain. 

It is recommended that in the future, alternative evaluation 
instruments be explored. The current evaluation checklist relif>.s 
on teachers' appraisals of students. Although such appraisals 
can yield useful information, they may also be influenced by 
factors unrelated to the project, for instance, the teachers' 
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TABLE 1 

hildren's Mean Raw Scores,* on the 
Developmental Checklist 
Brooklyn Prekindergarten Center, 1987-88 



Pretest Mean Postt^gt Mg^n Mean Gain 

Raw c>ercent Raw Percent Raw Percent 
Score Correct Score Correct Score Correct 



Students 

Meeting Criterion 106.0 57.9% 159.7 37.3% 53.7 29.3% 
(N=57) 

Studerts Not 

Meeting Criterion 129.0 70.5 154.7 84. b 25.7 14.0 

(N=3) 

Total 

(N=60) 107.2 58.6 159.5 87.2 52.3 28.6 



"Perfect raw score=183. 

* Ninety-five percent of participants met or surpassed the 
project-set criterion of a 20 percent increase. 
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increased familiarity with the child over the course of 

the program. This is especially true for the first part of the 

checklist, evaluating social behavior. 

In addition, it is difficult to attribute participants* 
improvement solely to the program without a comparison group of 
children who did not participate in the program. The age period 
of three to four is a time of growth, both physically and 
mentally, for all children, and although a prekindergarten 
program may help promote that growth, it is not possible to 
examine its impact beyond normal development without a comparison 
group. 

Finally, the present objective should be revised in more 
concrete terms. At present, the objective is ambiguously worded, 
stating that "participants will achieve a 20 percent gain." It 
is unclear whether this refers to each child individually or to 
the average score received by the group of children. It is not 
reasonable to expect all children to receive gains. Therefore it 
is recommended that the objective be revised to a specific 
quantitative goal, for example, "80 percent of participants will 
achieve a 20 percent gain." 



EVALUATION 
DEVELOPMENTAL CHECKLIST 
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Child_ ^Date of Birth 

Pre-kindergarten Class^ 



Pre-kindergarten Teacher 



Not Some of Most of 
yet the time the time 



Personal and Social Behavior 



1* 


Buttons coat or jacket without 




assistance 




Zips coat or jacket without 




assistance 


3. 


Removes and puts on rainboots 




without assistance (not tied) 


4. 


Fastens own shoes (not tied) 


5. 


Knows full name 


6. 


Knows home adaress 


1. 


Replaces toys and materials 


8. 


Evaluates own behavior 


9. 


Makes friends with other children 


10. 


Snows reasonable self control 


11* 


Follows class rules 


12. 


Accepts cnanges in routines 


13. 


Shares with other children 


14. 


Knows and uses names of adults 




in classroom 


15. 


Tries new things 


16- 


Is able to take turns 
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Not Some of Most of 
yet the time the time 



Physical Development 

17, Uses large muscles to: 
hop 
jump 
run 

gallop 
throw a ball 



18, Uses small muscles to: 
cut with scissors 
paste 

draw with crayons 
manipulate toys 
play with puzzles 
work with clay 



19. 


Walks up and down steps using 
two feet alternately 




Climbs, slides r swings on outdoor 
requipment 


21. 


Is able to place pegs in holes 


22. 


Strings one-inch beads 


23. 


Erects simple- block structures 


Intellectual development 


24. 


Expresses curiosity 


25. 


Recalls a sequence of events 


26. 


Recognizes likenesses and dif- 
ferences in a variety of materials 


27. 


Understands basic concepts: 

up/dov/n 

sweet/sour 

more/less 

big/little 

under/over 

soft/hard 
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Not Some of Most of 
yet the time the time 



2 3. 


Counts from one to five 


29. 


Participates in dramatic play 


30. 


Speaks in an understandable manner 


31. 


Relates incidents in simple 




1 r5 no n Arrp 


32 


tisps spntpnops avpraaina f iv© 




or six words 


O -J • 




OH • 




35. 


Talks freely to adults in room 


36. 


Identifies common objects 


37 . 


Rt*tplls a storv 


38. 


Shows un interest in books and 




stories 


39. 


Recites simple poems ana 




finger plays 


40. 


Memorizes and sings simple songs 


41. 


Participates in group discussions 




and meetings 


Creative Expression 


42. 


Sings songs 


43. 


Experiments with paints 


44. 


Dramatizes simple stories and poems 
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Not Some of Most of 
^et the t ime the time 



45. 


Explores the sounds of rhythm 




instruments 


46. 


Responds to music through movement 


47. 


Models witb clay 


48. 


Uses a variety of tools in drawing 
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PUBLISHING ACTIVITY CENTER, 1988-89 

School-Community Education Program 
Program Administrator: M. Morris Speiser 
Project Coordinator: Robert Fixler 

Prepared by: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The Publishing Activity Center is a career education and 
industrial arts project designed for elementary school pupils in 
Community school Districts (C.S.D.s) 14, 20, 21, and 22. The 
project seeks to foster career awareness among students, and 
motivate them to improve their reading and writing abilities. In 
1979, the State, Education Department validated the Publishing 
Activity Center as an Exemplary Project. 

In 1988-89, the program served some 1,450 kindergarten 
through grade six students. Selected pupils represented a cross 
section of the student population at each district. School 
principals and professional staff selected an appropriate sample 
of students for each site, according to locally determined 
sampling methods. 

There are some 20 Publishing Activity Centers established in 
selected schools. Each center is equipped with a binding 
machine, typewriters, mimeographs, and other publishing 
equipment. Three educational assistants, working under the 
supervision of a licensed teacher, visited these centers. Project 
activities were organized to promote students' creative writing 
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and artwork. Students wrote, typed, and published their own 
materials. Other activities Included instruction in the use of a 
variety of publishing equipment and an introduction to publishing 
careers. The New York State Legislature contributed $73 thousand 
to fund the program. 

The project objective was for 75 percent of participants to 
achieve a 25 percent gain in their knowledge of publishing 
equipment and procedures as measured by a project developed test. 
EVALUATION METHODOLOGY 

Project impact was assessed by analyses of student scores on 
a project-developed test that was designed to measure students' 
knowledge of publishing equipment, production processes, and 
awareness of careers in publishing. The test consisted of 20 
multiple-choice items and potential scores ranged from zero to 20 
(see Appendix A). The test was administered and scored by 
project paraprofessionals. Mean scores were computed for each 
grade and for all participants, across grades. 
FINDINGS 

Scores on the test were submitted for a random sample of 394 
participants in grades two through eight. A breakdown by grade 
is given in Table 1. A maximum of 20 points could be scored on 
this test. Mean pretest raw scores for each grade ranged from 
6.2 to 12.8 points, with an overall mean pretest score of 10.9 
points (54^.5 percent correct). Mean posttest scores ranged from 
13.3 to 17.3 points, witb an overall posttest mean of 14.4 points 

2 
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TABLE 1 

Students' Mean Raw Scores* on a Project-Developed Test, 

by Grade 

Publishing Activity Center, 1988-89 







Pretc 


Mean 


Posttest Mean 


Mean Gain 


Grade 


N 


Raw 
Score 


Percent 
Correct 


Raw 
Score 


Percent 
Correct 


Raw 
Score 


Percent 
Correct 


2 


60 


8.2 


41.0% 


14.6 


73.0% 


6.4 


32.0% 


3 


26 


6.2 


31.0 


17.3 


86.5 


11.0 


55.0 


4 


100 


9.9 


49.5 


14.9 


74.5 


5.0 


25.0 


5 


69 


12,3 


61.5 


13.3 


66.5 


1.0 


5.0 


6 


122 


12.8 


64.0 


14.1 


70.5 


1.4 


7.0 


7 


8 


13.4 


67.0 


13.2 


66.0 


-0.2 


-1.0 


8 


9 


13.7 


68.5 


13.4 


67.0 


-0.3 


-1.5 


TOTAL 


394 


10.9 


54.5 


14.4 


72.0 


3.5 


17.5 



"Perfect raw score = 20. 



* Overall mean gain was 17.5 percent. 

* Third graders outperformed students in oth<iir grades* 
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(72 percent correct). Mean gains ranged from 1.0 to 11 points, 
with an overall mean gain of 3.5 points (17.5 percent). 

Table 2 presents the percentage of students who met the 
project-set criterion of achieving a gain of at least 25 percent 
from pretest to posttest. Overall, 34 percent of participants 
met this objective. All of the third grade students sampled met 
or surpassed the project-set criterion. No students sampled from 
grades seven or eight met the criterion. 
CONCLUSIONS AND RECOMMENDATIONS 

In 1988-89, the Publishing Activity Center project was not 
successful in meeting its objective of 75 percent of the students 
achieving a 25 percent gain at posttest. Overall, only 34 
percent of the students met this criterion, but there was a large 
variation in scores between grades. One hundred percent of the 
third grade students met or surpassed the project-set criterion, 
whereas no seventh or eighth grade students met this criterion. 
An examination of the data reveals that students in grades seven 
and eight scored the highest pretest scores with 67 to 68 percent 
correct responses. Such high pretest scores preclude a 25 
percent increase on posttest (ceiling effect) and suggest that 
the test is too easy for these students. It is recommended that 
project staff revise the testing instrument to accommodate 
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TABLE 2 

Percentage of Students Meeting Project-Set Criterion* 
on a Project-Developed Test, by Grade 
Publishing Activity Center, 1988-89 





Grade 


N 


Meetina Criterion 
N % 



2 60 

3 26 

4 100 

5 69 

6 122 

7 8 

8 ' 9 
TOTAL 394 



41 


68.3% 


26 


100.0 


47 


47.0 


8 


12.0 


11 


9.0 


0 


0.0 


0 


0.0 


133 


33.8 



^Seventy-percent of participants will achieve a gain at posttest 
of at least 25. 

Overall, 33.8 percent of participants met the project- 
set criterion. 
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these students in the higher grades. One possible way. to 
accomplish this is to develop different tests for e^ch grade 
level. Another possibility is to expand the test varying the 
difficulty level of the questions to accommodate all grade 
levels. 
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EL£M£f(TARY SCHOOL CAREER AWARENESS PUBLISHING ACTIVITY CENlff^^^^JIji 
Pre ani ^ost Test 



Name 
School 



Date ^ 
Grade 



just do ^our best! 





Tnis machine makes many copies. 



Tnis 



is a A) stencil maker B) washing machine C) mineo D) Yarltyp 



2. 




This machine makes one 
szencil from the original 



This is a A) stencil maker 



D) binder C) proof -press 0) radio 
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3. 




We can use this machine to write w 
correct our work when we don't usii 
pencil 



This is a A)ofr-set press B) calculator C)Line-o-scribe D) Computer 



4. 




This piece of equipment is xised to 
make copies without a stencil or 
piinting plate. 



This is a A) OfF-set press B) Calculator C) Copy Machine D) Binder 



5. 




This piece of equipment is used 
when large numbers of high quality 
copies are needed. 



This is a A) Off-set press B) Scrabble Board C) Computer D) Typewriter 



KOW DO \VE USE OUR MACKT\n^S? 



6. Can you mix dr'Terent sizes of type in one line when you use a word processing 
program? 

A) Yes B) Sometimes C) Never D) every other time 



7. the door to the disk drive is not shut tightly, what will happen. 

93421 

n '5!f h'I'^m/':^^ 1^ '^^ computer can't save or retrieve information 
C) The Disk will np D) The computer will produce a double image 

8. Do the holes at the top of the stencil fit the notches on the mimeo machine? 
A) every other time B) sometimes Oyes D) never 

■ 9. How do you set the Xerox for 15 copies? 
A)150 B)015 0)051 D)105 

10. Which is a sample of 24pt type? 

A) T, B)Type OType D)T3rpe 

11. In our pubHshing activity center, we use ^to prepare our layout sheet. 

A) rubber cement B) Elmer s glue C) Crazy glue D) scotch tape 

draw^iT '° corrections on our ^^Titings and 

A) corrsction fluid B) Crazy glue C) Wite-out D) masking tape 

13. ^^^lich is a sample of Italic type? 

A) Type B)Iv£s C) Tj3>a D)2>pe 

14. Photographs must be converted to ____before they are printed. 
A) slides B) half-tones C) Fonts D) Platens 

41 



16. Which is a sample of Bold type? 
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A) Type B) Type OTyps D) TVpe 

HERE ARE SOIME CAREER QUESTIONS 

16. I seek out the news story wherever I go, so that everyone is in the know. 

WHO AM I? 

A) a reporter B) an editor C) a proofreader D) a publisher 

17. My hands are always full of ink. 

WHO AM I? 

A) a newscaster B) an astronaut C) a printer D) a publisher 

18. I deliver newspapers all over town. 

In all kinds of weather, Im always around. 

WHO AM I? 

A) a secretary B) a printer C) a newsboy D) a shop teacher 

19. Any iinisspelled words create a fuss, so correcting them is always a must. 

« 

WHO AM I? 

A) an editor B) a newscaster C) a proofreader D) a milkman 

20. I group the pages together so that, the reader knows where they are at. 
VmO AM I? 

A) a collator B) a reporter C) a printer D) a newscaster 
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BASIC SKILLS AFTER-SCHOOL PROGRAM, 1988-89 

School-Community Education Program 
Program Administrator: M. Morris Speiser 
Project Coordinator: Heywood Feierstein 

Prepared by: 
Office of Research, Evaluation, and Assessment 
Instructional Suprort Evaluation Unit 
New York City Public Schools 

PRGJECJ DESCRIPTION 

The Basic Skills After-School program is designed to 
prepare pre-kindergarten children from Community School District 
(C.S.D.) 17 for formal school. The project seeks to provide 
stimulating instruction in reading, mathematics, and language 
skills, other aspects of the project involve providing English 
as a Second Language instruction to limited English proficient 
(LEP) participants, and the promotion of p?»rent involvement in 
the program. 

In 1988-89, 22 children participated in the project. The 
children were selected according to criteria from needs 
assessments, including family eligibility for free lunch, the 
school's knowledge of the family ^s needs, and recommendationc 
from school and community agencies. Interviews were also used 
to select LEP participants. 

Instructional activities emphasized the development of 
language, letter, and number recognition, motor skills, and 
social interaction. Classes were held for two hours a day, 
three days per week. The project objective was for participants 
to demonstrate a 25 percent improvement in readiness and 
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developmental skills as measured by the developmental checklist. 

The voluntary component for parents involved workshops 
designed to help them better understand their children's 
development and learn techniques to support school performance. 
Project staff consisted of para -professionals, two pe^t-time 
teachers, and one part-time supervisor. The New York State 
Legislature contributed $9 thousand to fund the program. 
EVALUATION METHODOLOGY 

The evaluation focused on analysis of participants' pre- 
and posttest scores on the developmental checklist (see Appendix 
A). The developmental checklist assesses four levels of child 
development: personal an^ social behavior, physical development, 
intellectual development, and creative expression. Teachers 
rate participants on a scale of 1 to 3, with l=the child does 
not exhibit this behavior, to 3=the child exhibits this behavior 
most of the time. 
FINDINGS 

Complete test scores for 22 children were submitted on two 
separate data retrieval forms. These scores, distinguished as 
pre-kindergarten classes one and two, are reported in Table 1. 
Overall mean gain was 34.7 raw points. Pretest mean raw score 
was 124 points (66.7 percent correct responses), and posttest 
mean raw score was 158.7 points (85.3 percent correct). 
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TABLE 1 

Pre-School Children's Mean Raw Scores* on 

the Developmental Checklist 
Basic Skills After-School Program, 1988-89 



Pretest Mean 
Raw Percent 



Posttest Mean 
Rav. Percent 



H?an gain 

Raw Percent 



Class 


N 


Score 


Correct 


Score 


Correct 


Score 


Correct 


Pre-K (1) 


11 


117.9 


63.4% 


155.5 


83.6% 


37.6 


20.2% 


Pre-K (2) 


11 


130.1 


69.9 


162.0 


87.1 


31.9 


17.2 


TOTAL 


22 


124.0 


66.7 


158.7 


85.3 


34.7 


18.7 



•perfect raw score=l86. 

* Overall, mean gain was 34.7 raw points. 
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Table 2 shows the ^/ercentage of participants who achieved 
the program set objective. Overall, 13.6 percent of the 
children achieved a 25 percent improvement in readiness and 
developmental skills from pretest to posttest. 
CONCLUSIONS AND RECOMMENDATIONS 

The evaluation findings indicate that the Basic Skills 
After-School program was not successful in 1988-89. Only 13.6 
percent of participants achieved a 25 percent improvement in 
readiness and developmental skills. One explanation for these 
findings is the ceiling effect. Those participants who did not 
meet the objective scored a mean pretest score of 78 percent. 
This left very little room for a twenty percent increase. It is 
possible that these high pretest scores are a result of the 
subjective nature of the developmental checklist. 

It is recommended that in the future, alternative 
evaluation instruments be explored. The current evaluation 
checklist relies on teachers' appraisals of students. Although 
such appraisals can yield useful information, they can also be 
influenced by factors unrelated to the project, for instance, a 
teacher's increased familiarity with the child over the course 
of the program. This is especially true for the first part of 
the checklist, which evaluates social behavior. 

In addition, it is difficult to attribute participants' 
improvement solely to the program without a comparison group of 
children who did not participate in the program. The age period 

4 
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TABLE 2 

Participants' Achievement of Stated Program Objective* 
on the Developmental Checklist 
Basic Skills After-School Program, 1988-89 





Grade 




Total Number of 
Children Who Took 
Both the Pretest 
and Posttest 


Number of 
Achieving 
Gain From 
Posttest 


Children 
a 25% 
Pre- to 


Percentage 
Achieving 
Program 
Objective 


Pre-K 


(1) 


11 


1 




9.1% 


Pre-K 


(2) 


11 


2 




18.2 


TOTAL 




22 


3 




13.6 



•program objective: Participants will achieve a 25 percent 
improvement in readiness and developmental skills* 



* Overall, 13.6 percent of participants met the project 
set criterion. • 
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of three to four is a time of growth, both physically and 
mentally, for all children, and although a prekindergarten 
program may help promote that growth, it is not possible to 
examine its impact beyond normal development without a 
comparison group. 

Finally, the present objective should be revised in more 
concrete terms. At present, the objective is ambiguously 
worded, stating that "participants will achieve a 20 percent 
gain." It is unclear whether this refers to each child 
individually or to the average score received by the group of 
children. It is not reasonable to expect all children to 
receive gains. Therefore it is recommended that the objective 
be revised to a specific quantitative goal, for example, "75 
percent of participants will achieve a 20 percent gain." 
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EVALUATION 
DEVELOPMENTAL CHECKLIST 


APPENDIX A 
93426 


Child Date of 


Birth 


Pre 


-kindergarten Class 




Pre 


•kindergarten Teacher 






Not 
yet 


Some of Most of 


Personal and Social Behavior 




1. 


Buttons coat or jacket without 
assistance 




2- 


Zips coat or jacket without 
assistance 




3. 


Removes and puts on rainboots 
without assistance (not tied) 




4. 


Fastens own shoes (not tied) 




5. 


Knows full name 




6. 


Knows home adaress 




1. 


Replaces toys and materials 





8. 


Evaluates own behavior 




9- 


MaKes friends with other children 




10. 


Snows reasonable self control 




11. 


Follows class rules 




12. 


Accepts cnanges in routines 




13. 


Shares with other children 




14. 


Knows and uses names of adults 
in classroom 




15. 


Tries new things 




16. 


\Is able to take turns 

> 
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Not Some of Most of 
yet the time the time 



Physical Development 



17. 


Uses large muscles to: 

hop 

jump 

run 

gallop 
throw a ha 1 1 


18. 


Uses small muscles to: 
paste 

draw with crayons 
man i Du 1 ;5 1" p hnv^ 

play with puzzles 

work wit"h r*^ 
y9\jitt\ wxuij ^xdj^ 


19. 


Cliv •awwn out^Uo UaXriM 

two feet alternately 


20. 


Climbs, slides, swings on outdoor 
requipment 


21. 


Is able to place pegs in holes 


22. 


Strings one-inch beads 


23. 


Erects simple block structures 


Intellectual development 


24. 


Expresses curiosity 


25. 


Recalls a sequence of events 


26. 


Recognizes likenesses and dif- 
ferences in a variety of materials 


27. 


Understands basic concepts: 



up/down 
sweet/sour 
more/less 
big/little 
under/over 
soft/hard 
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Not Some of Most of 
yet the time the time 



45. 


Explores the sounds of rhythm 
instruments 




46. 


Responds to music through movement 




47. 


Models with clay 




48. 


Uses a variety of tools in drawing 
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Not Some of Most of 
yet the time the time 



45. 


Explores the sounds of rhythm 
instruments 




46. 


Responds to music through movement 




47. 


Models with clay 




48. 


Uses a variety of tools in drawing 
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#5001-48-93437 

TRAVELING BODY SHOP, 1988-89 

School-Community Education Program 
Program Administrator: M. Morris Speiser 
Project Coordinator: Robert Feinstein 

Prepared by: 
Office of Research, Evaluation, and Assessment 
Instructional Support Evaluation Unit 
New York City Public Schools 

PROJECT DESCRIPTION 

The Traveling Body Shop provides intensive instruction on 
the human body functions and on health maintenance and 
improvement to elementary school pupils in Community School 
District (C.S.D.) 31. In 1988-89, some 1,800 fourth and fifth 
graders from 18 schools participated in the program. In 
selecting these schools, priority was given to those which were 
located in low-income areas, had experienced cutbacks in health 
services, and were not participating in other School-Community 
Education projects. 

Project staff consisted of one full-time and one part-time 
teacher. They visited every participating class for two weeks in 
each school where they held 75-minute classes. Instructional 
activities varied according to grade level, focusing on the human 
body functioning and maintenance, health, ami nutrition. 
Students were also taught to measure lung capacity, blood 
pressure, and pulse rates. Classroom teachers received follow-up 
instruction in order to build on the lessons and activities of 
the two weeks of intensive instruction. The project objective 
for 1988-89 was for 75 percent of the participating pupils to 
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improve their knowledge of the major body systems, nutrition, and 
health, by 25 percent or more at post test, as measured by a 
project-developed test. The New York State Legislature provided 
$97 thousand to purchase instructional supplies and cover 
traveling expenses. 
EVALUATION METHODOLOGY 

Project impact was assessed by an analysis of students* 
scores on a program-developed test administered at the beginning 
and end of project activities (see Appendix A). The test was 
given on a pretest and posttest basis to determine whether the 
performance of students improved as a result of their 
participation in the project. Project staff collected data from 
classes at randomly selected schools* Approximately ten percent 
of the total students at each grade level were selecced as sample 
crroups . 

FINDIN GS 

Pretest and posttest scores were reported for 200 students. 
Table 1 presents evaluation findings by grade. Overall, mean 
pretest raw score was 9.4 points (47 percent correct responses), 
mean posttest raw score was 15.8 points (7S percent correct 
responses), for a mean gain of 6.4 points or 32 percent. Fifth 
graders achieved the largest mean gain. 

Table 2 presents the number of students who met the project- 
set criterion of achieving a 25 percent gain at posttest.. 

2 
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TABLE 1 

Students' Heau Raw Scores* on a Project-Developed Test, 

by Grade 
Traveling Body Shop, 1988-89 







Pretest Mean 


Posttest Mean 


Mean Gain 


Grade 


Raw Percent 
N Score Correct 


Raw Percent 
Score Correct 


Raw Percent 
Score Correct 



4 


32 


8.4 


0% 


14.6 


73.0% 


6.2 


31.0% 


5 


168 


9.6 


48.0 


16.0 


80.0 


6.4 


32.0 


TOTi^T. 


166 


9.4 


47.0 


15.8 


79.0 


6.4 


32.0 



•perfect raw score=20. 

Participating students showed an overall mean gain of 
32 percent points. 
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TABLE 2 

Percentage of Students Meeting Project -Set Criterion/ 

by Grade 
Traveling Body Shop, 1988-89 





Grade 


N 


Meetino Criterion 
N % 


4 


32 


22 68.7% 


5 


168 


128 76.2 


TOTAL 


200 


150 75.0 


'Seventy-f ive percent of participants will achieve a 25 percent 
gain on a project-developed test. 


Students in 
for success. 


grades five met the project-set criterion 
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Overall, 75 percent of the students met or surpassed the project- 
set criterion. Fifth grade students demonstrated a higher 
success rate than fourth grade students. 
CONCLUSIONS AND RECOMMENDATIONS 

In 1988--89, the Traveling Body Shop project wblp partially 

#^ 

successful in meeting its objective. Overall, 75 percent of 
participants achieved at least a 25 percent gain in their 
knowledge of the major body systems^ nutrition, and health, at 
posttest. Although the program was successful overall, an 
examination of the data reveals that fourth grade students did 
not perform as well as fifth grade students. Only sixty-nine 
percent of the fourth grade students sampled met or surpassed the 
project-set criterion. In 1987-88 the results were just the 
opposite, with fourth grade students receiving higher gains than 
fifth grade students. It is unclear what factors could have 
caused this difference between grades. In th^ future, project 
staff may wish to examine the content and presentation of the 
lessons at various sites to determine which factors may be 
affecting students' performance. 
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* APPENDIX P 

Board of Educalion, Q'tywide Umbi-ella Bureau. 347 Baltic Street *Rm 206, Brooklyn, New York 11201 93437 




BRAIN STRAIN 



1. Breathing rates vary according to the Individual and his or her 
activities. Your breathing is probably slowest when you are: 

a) running b) jumping c) sitting d) walking 



Your vital lung capacity is the amount of air you can force out ofyour lungs. The amount of 
air can be measured by blowing into a device called a spirometer. Which of the following 
pictures is a spirometer? 





3. Calcium is the mineral needed for the growth of bones and teeth. Who would have a 
greater need for calcium? 



a) adult 



b) young child 



d) same for all 



c^senior citizen 

4. You can make your heart and lungs work better by doing certain activities. Which activity 
listed below would most benefit your heart and lungs? 

a) lifting weights b) jogging c) playing baseball d) playing football 

5. The Heimlich Maneuver has been used successfully to aid people who are choking. In this 
technique, the object blocKing the windpipe may be removed by pressing firmly on a larg^ 



smooth muscle called the: 
a) tricep ' '* ' 5) bleep 



c) diaphragm 



d) heart 



To which food g r oup do gg gs b elong? 
a) milk b) meat 



c) fruits & vegetables d) grain 



7. Youf body has two blood pressure rates. One rate represents each beat or pump of your 
heart. The other rate represents between beats or when the heart is resting. The name of 
the blood pressure when ihe heart is resting is called: 

a) base rate pressure b) systolic pressure c) diastolic pressure ' d) venal pressure 

8. Every pump of your heart sends out a surge of blood. This surge passes along each portion 
of your blood vessel and forces the blood vessel wall outward. This '.ctlon of stretching and 
springing back of the blood vessel wall produces a: 

a) heartbeat b) pulse c) breath d) reflex 

9. Your body needs nutrients and oxygen transported by your blood. Feeling your pulse is a 
method to determine If your blood Is moving. V/hich picture below shows the correct way to 
take your pulse? 




10, A sign of physical fitness Is the time it takes the pulse rate to return to normal. A person in q 
good physical condition has a puise rate that, ' ^ 



Q a) quickly 

lERIC— ^ 



b) slowly 



at, after exercise, will return to normal: ^^^y^ 
c)jittle by little d) none of these^ vA}^ 
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11. Blood circulatts around tha" body through thou«ands of miles of vessels. 
The largest of your blood vessels are called 

-cl-*- '-^ 1 —.-1 ' .*--- — * ••• _ . 

al veins*" " ' b) capillaries ^ ^ ^ c) tracheas d) arteries 

12. The food group that gives us the greatest amount of vitamins A and C is: 

a) milk b) meat c) fruits i vegetables d) grain 

13. The..nu»be3^1 cause of death in the. U nited States is; ^ 

a) cancer 



b) heart disease 



c] aids 



d)"none of these 



14. 



IS. 



... \ ^ . 

Fat is' necessary to carry certain vitWins in your body. Hovever/ sone' types of 

fat can be vexy dangerous for your health. A fat )cnovn^to^ increase your jdiances 

of. d^eveloping heart disease is: r-*"*:i;." 

a) ^coconut oil '* *b) lard c) palm oil . d) a].l of the^u 

The pumping of your heart sends out a surge of blood causing pressure on the blood 
vessels.- Your blood pressure can be measured by using an instrument called a 
sphygmomanometer. Which picture below is a sphygmome:'ometef? 





16. You are what you eat therefore, it. is important to read the list of ingisdients on the : 
package. The ingredients on packaged food products are listed: . • v • .- v ^rz'az -j-.- J' 

a) by amounts b) alphabetically c) in any order d}with naturalitems first 

17. Your pulse recovery rate is a simple method of measuririg your' physical fitness. When 
judging your physical fitness improvement you* should compare'yourself to: 5 - 



a) a trained athlete b) yourself 



c) an adult. 



d) another student 



18. You hav2 over 600 muscles in your body. Since these muscles can only move one way. 
they usually work in pairs. To keep our muscles working properly, it is important that we get: 

a) proper diet^ b) proper rest c) proper exercise d) ail of these . ..^ 

19. Which part of the air we breathe combines with the food we eat to give us energy? 
a) carbon dioxide b) oxygen c) helium d) nitrogen 

20. The hunian body is incredible! The' best way to take care of your body is b^' developing and 
maintaining good health habits. You should start to develop these good health habits.*: ; 

a) wheri'ybu feel sick b) today c) tomorrow d) when the doctor tells you to. 
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